8-9 t/s并没有完全发挥8060S的潜力，llama.cpp的llama-server有一个小问题导致server配speculative decoding时性能欠佳（与具体硬件无关）：https://github.com/ggml-org/llama.cpp/issues/12968
动手简单修一下这个问题之后Qwen 2.5 72B iq4_xs + 1.5B draft在acceptance rate理想时可达到10-12 t/s左右
https://github.com/hjc4869/llama.cpp/commit/0b32f64ffbe973e99e0dc7097be31d4d966d476e
BY David's random thoughts
The STAR Market, as is implied by the name, is heavily geared toward smaller innovative tech companies, in particular those engaged in strategically important fields, such as biopharmaceuticals, 5G technology, semiconductors, and new energy. The STAR Market currently has 340 listed securities. The STAR Market is seen as important for China’s high-tech and emerging industries, providing a space for smaller companies to raise capital in China. This is especially significant for technology companies that may be viewed with suspicion on overseas stock exchanges.
Since the violent storming of Capitol Hill and subsequent ban of former U.S. President Donald Trump from Facebook and Twitter, the removal of Parler from Amazon’s servers, and the de-platforming of incendiary right-wing content, messaging services Telegram and Signal have seen a deluge of new users. In January alone, Telegram reported 90 million new accounts. Its founder, Pavel Durov, described this as “the largest digital migration in human history.” Signal reportedly doubled its user base to 40 million people and became the most downloaded app in 70 countries. The two services rely on encryption to protect the privacy of user communication, which has made them popular with protesters seeking to conceal their identities against repressive governments in places like Belarus, Hong Kong, and Iran. But the same encryption technology has also made them a favored communication tool for criminals and terrorist groups, including al Qaeda and the Islamic State.
David& 39;s random thoughts from id